parallel distribut data manag abstract knowledg discoveri tool techniqu increas number scientif commerci area analysi larg data set larg data repositori coupl geograph distribut data user system combin technolog implement high perform distribut knowledg discoveri system hand comput grid emerg promis infrastructur high perform distribut comput paper introduc softwar architectur parallel distribut knowledg discoveri pdkd system built top comput grid servic provid depend consist pervas access high end comput resourc propos architectur grid servic defin set addit layer implement servic distribut knowledg discoveri process grid connect sequenti parallel comput dwarehous visual olap analysi trajectori data dwarehous visual olap analysi trajectori data task dbase search recommend system gener pattern model mine binari dataset mine home public dresourc comput framework distribut data mine mine top dk pattern binari dataset presenc nois mine home public dresourc comput framework distribut data mine mine queri log propos method rank special categori time sensit queri year qualifi method adjust retriev score base rank function time stamp web document freshest document rank higher method base feedback control theori rank error adjust search engin behavior purpos simpl effect method extract year qualifi queri mine queri log time stamp recognit method consid titl url web document method test commerci search engin experi show approach significantli improv relev rank year qualifi queri exist method comparison fail cach content dbase queri robust effici imag retriev challeng design interest dbase distribut aggreg user system frequent spatio dtempor pattern trajectori data warehous cach algorithm similar search constraint dbase queri system exploratori pattern discoveri multi dset dht rang queri dynam data grid inform servic queri histori teach includ futur metric cach similar search similar search metric space gener paradigm applic field effect exploit content base imag retriev system shift target web scale dimens context import issu design scalabl solut combin parallel distribut architectur cach level end investig design similar cach work metric space answer exact approxim result exact match present cach cach return approxim result set qualiti guarante conduct test collect million high qualiti digit photo show propos cach techniqu signific impact perform cach text queri prove effect tradit web search engin process complex similar queri peer dto dpeer network trajectori data warehous store aggreg frequent st pattern scalabl distribut data structur multi dfeatur similar search spatio dtempor aggreg trajectori data warehous spatio dtempor aggreg trajectori data warehous approxim aggreg trajectori data warehous parallel mine frequent close pattern har modern comput architectur mine queri log optim index partit parallel web search engin respons time estim web servic mine queri log optim index partit parallel web search engin month life ege south south client side estim remot servic execut abstract case monitor control real physic instrument demand deep interact user servic virtual access instrument devic addit order realiz high interoper solut soa base web grid servic technolog adopt access servic perform internet web servic call remot invoc time critic order understand instrument control properli delai introduc wire serial deseri process unaccept paper present methodolog algorithm base factori analysi gaussian major previou servic execut time enabl estim gener remot method execut time suggest softwar architectur develop algorithm methodolog integr order automat profil end toend servic worth note propos valid suitabl benchmark extens test come real simul environ addit outcom paper realiz servic remot control monitor manag pool instrument devic trajectori data warehous design issu case face integr instrument grid face integr instrument approxim mine frequent pattern stream approxim mine frequent pattern peer dto dpeer system discov resourc dynam grid instrument element grid compon enabl control remot instrument resourc discoveri system base forest tree converg grid peer peer world led solut effici solv problem resourc discoveri grid solut extens dht base network system flexibl case data present network dynam data valu chang frequent time common case kind data typic grid system cpu load queue occup paper present variat previou work base tree shape overlai network system rout index effici rout queri updat messag presenc highli variabl data system base twolevel hierarch network topolog tree topolog maintain lower level hierarchi main aim organ achiev simpler mainten graph topolog reach cost imprecis rout index built upper level graph hierarchi improv perform xml base technolog cach reus inform perform xml base technolog cach reus inform interact pattern mine relat databas cluster import tool data mine identifi major pattern trend supervis label data tree vector index effici rang queri dynam content peer dto dpeer network locat data peer peer network complex issu address protocol research area consid static content assum data system vari time paper describ data locat strategi dynam content network data locat exploit distribut index base bit vector index rout queri area system match found bit vector effici updat data modifi simul result show propos algorithm queri updat propag good perform larg network content exhibit high degre variabl privaci preserv web recommend system mine frequent close itemset dof dcore interact pattern mine relat databas cluster import tool data mine identifi major pattern trend supervis label data boost perform web search engin cach prefetch queri result exploit histor usag data articl discuss effici effect issu cach result queri submit web search engin wse propos sdc static dynam cach cach strategi aim effici exploit tempor spatial local present stream process queri sdc extract histor usag data result frequent submit queri store static read portion cach remain entri cach dynam manag replac polici queri satisfi static portion improv hit ratio sdc adapt prefetch strategi anticip futur request introduc limit overhead back end wse experiment demonstr superior sdc pure static dynam polici measur hit ratio achiev larg queri log vari cach paramet replac polici manag dynam part cach final deploi measur throughput achiev concurr version cach system test show sdc cach effici exploit thread search architectur softwar compon grid relat technolog enabl larg scale share resourc type envis futur applic complet built bottom fashion softwar compon deploi locat interconnect form workflow graph paper make propos design compon search servic enabl user locat compon deploi applic copyright john wilei amp son kei word grid comput workflow compon rank compon search pagerank fast memori effici mine frequent close itemset abstract paper present scalabl algorithm discov close frequent itemset lossless condens represent frequent itemset mine transact databas algorithm exploit divid conquer approach bitwis vertic represent databas adopt visit partit strategi search space base origin theoret framework formal problem close itemset mine detail algorithm adopt optim aim save space time comput itemset closur support main problem rais type algorithm multipl gener close itemset propos effect memori effici prune techniqu unlik previou propos requir set close pattern mine main memori techniqu permit visit partit search space mine independ order parallel test conduct publicli dataset show algorithm scalabl outperform state ofth art algorithm closet fp close case order magnitud importantli perform improv signific support threshold decreas ddimension hydrodynam interact supernova remnant shock isol cloud resourc discoveri dynam grid environ resourc discoveri grid environ critic problem typic grid system includ larg number resourc readili identifi access run applic tradit grid discoveri algorithm perform poorli scale grid enabl applic transpar queri set grid resourc peer peer argu suitabl distribut paradigm overcom issu scalabl discoveri system support discoveri resourc context dynam resourc inform paper propos system index discov grid resourc assum grid resourc character set attribut system queri resourc satisfi arbitrari rang condit attribut note tradit search techniqu directli appli case work static content exact queri simul result show system adequ degre scalabl distribut approxim mine frequent pattern prefac select paper europvm fmpi confer dci close fast memori effici algorithm mine frequent close itemset main problem rais frequent close itemset mine problem duplic detect paper propos gener techniqu promptli detect discard duplic close itemset keep main memori set close pattern approach exploit substanti perform benefit algorithm adopt vertic represent dataset implement techniqu depth close itemset mine algorithm experiment evalu demonstr algorithm outperform state art algorithm closet fpclose webdoc real dlife huge transact dataset kdci direct count iter wing parallel index web content statist properti transact databas complex common data mine task due unknown amount inform contain data mine pattern correl contain data resourc need extract confirm fact gener singl algorithm data mine task kind input dataset order achiev good perform strategi optim adopt dataset specif characterist typic distinct transact databas spars dens dataset paper frequent set count case studi data mine algorithm propos statist analysi properti transact dataset character dataset complex show character field perform predict optim algorithm gap constrain sequenc mine assign document identifi enhanc compress web search engin index grant effici access index kei issu perform web search engin wse order enhanc memori util favor fast queri resolut wse invert file index post list store sequenc gap differ success document identifi compress variabl length encod method paper describ lightweight cluster algorithm aim assign identifi document minim averag valu gap simul perform real dataset googl contest collect show approach obtain index depend gap encod chosen smaller built randomli assign document identifi show analyt empir complex algorithm linear space time assign document identifi enhanc compress fulltext indic silvestri perego orlando istituto isti consiglio nazional dell ricerch cnr pisa itali dipartimento di informatica universit di venezia mestr itali abstract index compress major issu field inform retriev system due impress figur involv web search engin wse compress index option anymor import index compress method design work invert file index method base assumpt post list store sequenc gap differ success document identifi compress carri variabl length encod method repres smaller number smaller number bit paper focus find encod method propos algorithm assign identifi document minim averag valu gap simul perform real dataset googl contest collect show approach obtain index depend gap encod chosen smaller built randomli assign document identifi show analyt empir complex algorithm linear space time assign identifi document enhanc cluster properti fulltext index web search engin provid larg scale text document retriev servic process huge invert file index invert file index fast queri resolut good memori util gap represent effect effici compress variabl length encod method paper propos evalu algorithm aim find assign document identifi minim averag valu gap enhanc effect tradit compress method ran test googl contest collect order valid techniqu propos experi demonstr scalabl effect algorithm propos algorithm sensibl improv compress ratio encod scheme search architectur grid softwar compon paper discuss challeng face design search servic locat softwar compon grid specif search engin reli heavili concept ecosystem compon basic consist extens marketplac concept grid ecosystem fact applic virtual environ live execut find compon refer cooper build larger complex system design parallel distribut web search engin paper describ architectur mose search engin scalabl parallel distribut engin search web mose specif design effici exploit afford parallel architectur cluster workstat modular scalabl architectur easili adjust fulfil bandwidth requir applic hand task parallel data parallel approach exploit mose order increas throughput effici commun store comput resourc recent advanc parallel virtual machin messag pass interfac kdci multi dstrategi algorithm mine frequent set introduct result global experi coffe break panel session session chair tba fast apriori implement ferenc bodon effici implement apriori eclat christian borgelt assistconf grid configur tool assist parallel program environ paper present assistconf graphic user interfac design configur assist program run grid platform assist softwar develop system base integr skeleton technolog program environ develop parallel distribut high perform applic main goal assist allow high level programm softwar product complex multidisciplinari applic perform portabl platform includ homogen parallel machin cluster beowulf system heterogen cluster comput grid assistconf configur assist program establish map program modul suitabl machin grid candid execut simplifi creation xml assist configur file give user machin execut final configur program grid hybrid strategi cach web search engin result work discuss design implement effici cach system aim exploit local present queri submit web search engin wse enhanc previou propos direct propos adopt hybrid strategi cach experiment demonstr superior hybrid strategi show advantag spatial local present wse queri log exploit sort adapt prefetch strategi schedul high perform data mine task data grid environ increasingli dataset data mine huge physic distribut distribut knowledg discoveri process data comput intens grid natur platform deploi high perform data mine servic focu paper core servic grid infrastructur concentr attent design implement special resourc alloc execut manag servic awar data sourc locat resourc data mine task alloc schedul decis basi perform cost metric model exploit knowledg previou execut sampl acquir estim execut behavior adapt resourc dawar mine frequent set perform algorithm mine frequent set transact databas sever depend specif featur data analyz architectur characterist comput platform main memori dramat chang runtim behavior algorithm paper present dci direct count amp intersect cient data mine algorithm discov frequent set larg databas ectiv address issu mention dci adopt classic level wise approach base candid gener extract frequent set hybrid method determin candid support innov contribut dci reli multipl heurist strategi emploi permit dci adapt behavior featur specif comput platform featur dataset mine result ectiv mine short long pattern spars dens dataset larg amount test conduct permit state dci sensibl outperform state ofth art algorithm synthet real world dataset final discuss parallel strategi adopt design pardci distribut multi thread implement dci effici parallel distribut algorithm count frequent set due huge increas number dimens databas cient solut count frequent set nowadai import data mine commun sequenti parallel algorithm propos case exhibit excel scalabl paper present pardci distribut multithread algorithm count occurr frequent set transact databas pardci parallel version dci direct count amp intersect multi strategi algorithm adapt behavior featur specif comput platform enhanc apriori algorithm frequent set count paper propos dcp algorithm solv frequent set count problem enhanc apriori goal optim initi iter apriori time consum dataset character short medium length frequent pattern consid main improv regard innov method store candid set item count support exploit effect prune techniqu significantli reduc size dataset execut progress integr hpf skeleton base parallel languag hpf programm express data parallel comput portabl high level wide accept import parallel applic effici implement pure data parallel paradigm applic singl data parallel program profit subdivid comput data parallel piec piec run concurr oper exploit task parallel paper discuss integr hpf skie skeleton base coordin languag implement top mpi messag pass interfac permit describ complex comput parallel structur show hpf insid common form parallel pipelin processor farm present experiment result sampl applic implement issu design fo intens data mine applic cluster workstat paper investig scalabl implement core intens data mine algorithm aordabl parallel architectur cluster workstat order valid approach mean algorithm dm cluster algorithm test case integr task data parallel taskhpf applic exhibit larg amount potenti parallel exploit data task level paper applic structur ensembl independ data parallel hpf modul hpf task assum hpf task interact static predict structur describ taskhpf coordin languag user dene hpf task clean input output interfac program global interact structur task declar examin small applic discuss benet approach show taskhpf program translat ecient messag pass code introduct applic show potenti exploit data task parallel data parallel character applic oper dierent part relat data set task parallel distinct part program proce ind coordin hpf program mix task data parallel experi applic eld deriv develop multidisciplinari parallel applic suggest model outer coordin level provid data parallel task run concurr cooper comput level coordin model easili express hpf high level data parallel languag model devis colt hpf coordin architectur layer support dynam creation concurr execut hpf task permit task cooper messag pass paper propos exploit colt hpf mean simpl skeleton base coordin languag sourc sourc compil dierent relat propos colt hpf portabl exploit commerci standard compliant hpf compil system physic applic test case approach present result exp mix data task parallel hpf pvm paper present colt hpf coordin layer task express hpf portabl coordin commun layer hpf task colt hpf implement top pvm suitabl mechan start run time instanc data parallel task disjoint group pvm virtual processor optim primit intertask commun data exchang distribut processor userspeci ed hpf direct colthpf run dtime support high dlevel dordin hpf task ion sda syntax similar hpf instanc sda encapsul distribut data method method exclus access encapsul data data parallel task start creat instanc specif sda inter task oper take place mean remot synchron asynchron method invoc note sda instanc start dynam call coordin task run time implement inter task commun control pass distribut data structur task includ remap need run time accomplish handshak protocol exchang distribut inform actual argument caller sda formal calle sda method note handshak protocol similar colt hpf protocol creat channel task final suppl effici run dtime support duniform parallel loop effici implement parallel loop distribut memori multicomput data parallel languag gener exploit static data layout static schedul iter iter execut cost vari consider unpredict processor assign work workload imbal mitig cyclic distribut data comput strategi solv load balanc issu worsen data local exploit paper present suppl suport parallel loop execut innov run time support parallel loop regular stencil data refer uniform iter cost suppl reli static block data distribut exploit local combin static dynam polici schedul uniform iter adopt static schedul polici deriv owner comput rule move data iter processor schedul data dparallel comput heterogen time dshare environ paper address problem load balanc data parallel comput heterogen time share parallel comput environ load imbal environ introduc capac processor popul comput share comput resourc user evenli partit code homogen system run perfect load balanc perform poorli processor speed load solv problem data parallel comput propos run time support parallel loop base hybrid static dynam schedul strategi support balanc processor workload dynam migrat comput slower faster processor requir priori knowledg rel speed result obtain experi conduct sgi crai ibm sp system show hybrid schedul strategi alloc sa coordin layer exploit task parallel hpf mpi dbase run dtime support coordin hpf task paper describ colt hpf mpi base run time support coordin concurr commun hpf task colt hpf conceiv compil high level coordin languag structur set data parallel hpf task popular form task parallel requir small run time support hpf compil colt hpf easili portabl compil system paper outlin design implement issu report result experi conduct sgi crai introduct due simplic data parallel program model high level parallel languag hpf design hide low level program detail programm concentr high level exploit data parallel compil manag data distribut translat annot sourc program spmd code explicit interprocessor commun comparison implement strategi nonuniform data dparallel comput support duniform parallel loop applic flame simul code paper present suppl support parallel loop execut innov run time support parallel loop regular stencil data refer uniform iter cost suppl reli static block data distribut exploit local combin static dynam polici schedul uniform iter adopt static schedul polici deriv owner comput rule move data iter processor load imbal occur suppl overlap commun comput reorder loop iter prefetch remot case workload imbal suppl approach valid experiment result obtain run multi dimension flame simul kernel node crai fed benchmark code synthet input data set built basi load imbal model compar result wi load balanc strategi iter parallel loop schedul effici templat implement distributedmemori multiprocessor iter parallel loop parallel loop nest sequenti loop present templat explicitli design smooth unbalanc processor workload deriv loop iter character highli vari execut time experi conduct show perform gain hpf languag support introduct effect exploit parallel loop kei achiev consider speedup scientif comput intens applic data parallel languag hpf reli user provid direct deriv distribut data owner comput rule determin static schedul parallel loop provid data distribut choic data distribut block cyclic programm note uniform problem imposs find good distribut scheme templat duniform parallel loop base dynam schedul prefetch techniqu paper present effici templat implement distribut memori multiprocessor uniform parallel loop loop independ iter character highli vari execut time templat reli static block distribut arrai data set exploit local hybrid schedul polici smooth uneven processor finish time initi adopt static schedul techniqu base owner comput rule workload imbal detect exploit dynam receiv initi techniqu move work unload processor prefetch reduc overhead due commun need monitor load move iter restor consist migrat data accur perform cost techniqu deriv allow templat compil gener balanc code uniform parallel loop experi conduct node exploit partial replic unbalanc parallel loop schedul multicomput problem schedul parallel loop iter oper larg arrai data structur character highli vari execut time unbalanc uniform parallel loop gener parallel loop implement templat messag pass distribut memori multiprocessor multicomput present assum imposs static determin distribut comput load data access templat exploit hybrid schedul strategi data partial replic processor local memori iter static schedul load imbal detect point effect dynam schedul techniqu adopt move iter node hold data commun need implement dynam load balanc overlap comput effect prefetch polici adopt templat scale know data summaris experi parallel program languag design structur high dlevel parallel languag paper present parallel program methodolog ensur easi program effici portabl program machin belong class gener purpos distribut memori mimd architectur methodolog base definit high level explicitli parallel languag call set static tool automat adapt program featur target architectur requir programm process activ actual parallel degre schedul interprocess commun featur adjust har specif target machin parallel hand express structur qualit hierarch composit restrict set languag construct form parallel frequent encount parallel applic effici implement eff compil geometr paradigm local commun paper present compil techniqu gener parallel code explicit local commun mesh connect distribut memori mimd architectur compil techniqu work geometr paradigm parallel comput effici compil structur parallel program distribut memori mimd machin synchron model static optim multicomput program databas analyz predictor overview strategi massiv parallel implement simul anneal integr tool physic databas design codasyl environ probabilist appraoch evalu data manipul algorithm codasyl data base environ 